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This report describes the developnent, revision, and implementation of a 
system for content analyzing students* narrative reports of critical/significant 
events that occurred during experiences in Education 450 and Education 451, 

The content analysis system Is a hierarcnical classification in which each 
event is classified in four ways: (1) type of experience, i.e., instructional 
strategy in which the event occurred, (2) type of event, i.e., teacher respon- 
sibility or area of teacher decision-making, (3) category of event, i.e., 
specific, situation or behavior under a type of event, and C4) affect of event, 
i.e., feeling expressed about the event. 

The first major content analysis using the system included 64 Critical 
Event reports from Education 450 and 103 from Education 451. Frequently 
reports included accounts of more than one critical event. Up to three events 
were coded f om each report, resj^lting tn a total of a count of 89 codeable 
events in Ed'.'cation 450 and 159 codeable events in Education 451. Vague or 
general descriptions were marked uncodeable. 

The results of the analysis showed that 85% of the critical events reported 
in Education 450 occurred in three types of experiences: field, microteaching, 
and reflective teaching. In Education 451 88% of the reported critical events 
occurred in the field. 

Two types of events, teaching (46%) and planning (30%), accounted for the 
major portion of type of events in Education 450. In Education 451 type of 
event was somewhat evenly distributed between four types of events: student 
characteristics (25%), teaching (24%), planning (22%), and classroom control and 
teacher-student relationships (20%). 

Of the 43 possible categories of events a few were classified frequently. 
In Education 450 high-frequency (5 or more times) categories included time for 
careful preparation, successful lesson, impact of evaluative feedback, and 
unexpected learner characteristics. In Education 451 13 categories were high- 
frequency categories, three of which matched 450 high-frequency categories. 

In both courses students expressed more positive than nc.itral or negative 
feelings; htMever^ the percentage of positive feelings in Education 450 (78%) 
was considerably higher than in Education 451 (56%), 

In conclusion the analysis shows a iiumbur of differences in types of 
experiences and events considered important to students in Education 450 and 
451. These differen'^as reflect the particular emphases in goals and experiences 
within each course and\ affirm the relevance of such experiences tr students. 
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Introduction 

One of the four major data components in the College of Education Student 
Information System (SIS) is narrative data. A medium through which narrative 
data is collected in the Professional Introduction course (PI) is the Critical 
Event Form (Appendix A). 

Students in PI 450 and 451 are asked to submit descriptions of specific 
professional experiences that have had particular importance or meaning to them. 
I.e., critical events. The student is first asked to write a low inference 
decription of the event. 'P'^n'a high inference judgement of the event is requested. 

This report details the development of a system for content analyzing PI 
students* reports of critical events. The major processes involved were: 

A. Development of an initial set of categories based on students' reports 
of events. 

B. Trial analysts of critical event reports using the initial categories 
and subsequent revision of the category system. 

C. Development of rater skill and determination of interrater reliability. 

D. Establishnent of procedures to be used in classifying events. 

E. Content analysis of a large sample of critical events. 

A. Development of an Initial Set of Categories 

Approximately 100 critical event reports were read and a listing cf the types 
of events contained in the reports was made. A synthesis of the varied descrip- 
tions resulted in a three-stage hierarchical classification. The first stage of 
the classification is type of experience . It is the type of teacher education 
experience or teaching strategy in which the described event occurs. Type of 
experience contained five major experience areas. An example is field experience. 
Type of experience subsumes the next stage of classification, type of event . 
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This Is the type of teacher responsibility or area of teacher decision-making. * 
There were five major event types; one example Is planning. Under each type 
of event a set of specific situations, behaviors, or outcomes was listed. A 
specific event Is classified as category of event . An example of a plam.ing 
category Is use of curriculum guides. A total of 29 categories were delineated 
under the five types of events. 

Generally, students* descriptions of events contained explicit references 
to their feelings about the events. In order to examine the dimension of feelings 
a fourth classification, affect of event , i.e., expression of positive, neutral, 
or negative feelings toward the specific event, was added to the analysis system. 
The four-part classification systan was formatted into a ?l Critical Event Content 
Analysis Form, ^ 

B. Trial Analysis Using the tnitial Set of Categories 

V 

A set of 50 randomly selected critical event reports were content ani^lyzed 
to try out the classification system. It was fourd that three additional types 
of experiences and 14 additional categories of events were needed. These were 
added to the system resulting in a total of eight types of experiences, five 
types of events, forty-three categories of events, and three levels of affect. 
To handle exceptions, an "other" category was added under each part of the classi- 
fication system except affect. The revised Content Analysis Fom is included 
In the report as Appendix B. 

C. Interrater Reliability 

Two raters independently analyzed three sets of 10 randomly selected critical 
event reports using the revised Content Analysis Fonn, The three sets of reports 
were labeled first, second, and third trial. Trials one and two were treated as 
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rater training sessions. Overall, the sessions yielded refinement of definitions 
and rater skill (see Table 2). 

In addition to agreement on the fwr parts of the classification, agreenent 
on the total number of events classified in each trial Mas calculated. Number 
of events reported on each critical event form varied, probably due to the 
complex nature of significant/critical learning events. It was decided to 
classify a maximum of three events from each report, 

/Agreement on the number of events classified in each trial was calculfited 
using analysis of variance. The mean manber of events classified by the two 
raters for each trial (set of 10 reports) is listed in Table 1. Ha statistically 
significant differences bej:ween the nunber of events classified by the two raters 
were found in the three trials (F values were .80, 1,27, and 1.00 respectively; 
an*F value of 4,35 was required for an .05 level of significance). 

Agreement on classifications of type of experience, type of event, category 
of event, and affect of event was defined as the percentage of time in which 
the same classification was assigned to pairs of identified events. When a 
second or third event that was classified by one rater had no pair from the other 
rater, the event was dropped from the comparison. The reason for dropping the 
unpaired event was that the difference in judgment was whether or not the report 
merited an additional classification. The difference in the number of events 
classified was not significantly different overall. 

Table 2 reports rater agreement for each trial on the four parts of the 
classification system. The degree of agreement was higher for trial 3 in three 
of four classifications. Agreement on affect, the most subjective element, did 
not change significantly over trials. 
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Table 1 



Mean jjmpber of Events Classified frwi each 
Critical Event Form by Two Raters 

Trial 1 Trial 2 Trial 3 

Rater 1 1.7 1.9 1.7 

Rater 2 2.C 1-5 2.0 

K « 10 forms In each trial 



Table 2 



Agreement Between Two Raters on Classification 

Critical Events 



(1) Type of experience 

(2) Type of event 

(3) Category of event 

(4) Affect of event 



Percentage of Agreement 



Trial 1 


Trial 2 


Trial 3 


94 


93 


100 


87 


93 


94 


67 


64 


81 


73 


71 


69 
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D. Procedural Guides Used In Classifying Events 

The following .guides were deieloged during the first and second reliability 

trials and served to provide consistency in the content analysis for the third 

reliability trial and the large sample anaftysis t^jt^llows: 

{1} Analyze events that are a significant part of the report. Usually 
a single sentence or a minor reference is not classified. 

(2) Classify up to three separate events fran each critical evint report. 

(3) Affect is to be coded in reference to each event; not in reference 

to each report form. 

(4) Reports that are general or vague will be marked non-codeable. 

(5) The classifications will be coded and transferred to optical scan 
sheets as follows: 

a. Identification Nianber - Social Security Number 

b. Special Codes: K 

Number of Critical Event Form 
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H — Course Number: 0=450, 1=451 
0 — Quarter form was collected: 

l«Summer, 2«Autumn, 3=Winter, 4»Spring 
P — Year form was collected: 
2»1982, 3=1983, 4=1984 
c. Item numbers will be used in groups of six as follows: 

Event #1 Event #2 Event #3 

column #1-2 Type of experience column #7-8 column #13-14 

3 Type of event 9 15 

4-5 Category of event 10-11 16-17 

6 Affect of event 12 18 



E. Content Analysis of a Large Sample 

The first major content analysis included 64 Critical Event reports from 
Education 450 and 103 from Education 451 collected at the end of Winter Quarter 
"•983. Students were asked to submit for analysis a report of the most significant/ 
critical event of the quarter. 

The reports were analyzed for number of events and frequency of type of 
experience, type of event, category of event, and affect of event. Descriptions of 
the analysis from the two courses are presented below. 

o 
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Himber of Reports and Events Analyzed 

Although the critical event form requests one critical event per form, 

♦ 

students often reported more than one. Table 3 shows the number of report 
forms and events analyzed for each course. The averaqe number of events classified 
per critical event form was slightly hlqher for Education 451 (1,6) than for 
Education 450 (1.4). 

Type of Experience In which Events Occurred 

The type of teacher education experiences In which the reported events 
occurred are shown In Table 4. In Education 450, 84.7% of the reported critical 
events occurred In three types of experiences. I.e., field, microteachinq, and 
reflective teaching. In Education 451, 87.7% of the critical events occurred In 
one type of experience. I.e., field experience. Non-codeable events were deleted 
from the remaining classifications, resulting In a count (K) of 89 events in 
Education 450 and 159 events in Education 451, 
Type of Event 

The type of events, i.e., teacher responsibility or area of decision making, 
that were reported as critical by teacher education students are shown In Table 5. 
Ift Education 450 the responsibility of teaching was critical, i.e., significant or 
meaningful, in 46.1% of the events. Planning, also a high frequency event, was 
reported in 30.3% of the events. 

The events of significance In Education 451 were somewhat evenly represented 
In four of the five major events: student characteristics (25.2%), teaching (23.9%), 
planning (22.0%), and classroom control' and teacher-student relationships (19,5%). 
The more even distribution is probably related to the qreater proportion of time 
spent In the field in Education 451 than in Education 450. 
Category of Event 

Although a total of 43 specific categories of situations and behaviors 
Within the five major types of events are included In the third stage of the 

ERIC 
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classification system, 60% of the critical events reported in Education 450 
occurred in only five categories. In Education 451 72% of the critical events 
occurred in 13 categories. Three of these frequently occurring categories are 
the same in both courses. A high-frequency category is one that occurred five 
or more times. See Table 6 for frequency of all 43 categories and Table 7 for 
htgh-frequency categories. 
Affect of Event 

A student's report of feelings about an event was classified into positive, 
neutral, or negative affect for each event. If a student Included more than one 
feeling of affect for an event, the concluding feeling was used for classification 
purposes. Although students were more positive than negative in both courses, 
more negative feelings were expressed in Education 451 than in Education 450 
(see Table 8). 
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Table 3 

f^ber of Reports and Events Analyzed In PI Courses 



Educ 4S0 Educ 451 

Number of Report Forms ^alyzed (N) 64 • 103 

Number of Events Classified (K) 91 163 

T Events per Report 1.4 1,6 



Table 4 



Type of Experience 
(Teacher Education Strategy ) 



Educ 450 Educ 451 





K 


% 


K 


% 


Field Experf«ince 


15 


16.5 


149 


87.7 


Microteachinq 


32 


35,2 


5 


3.1 


Reflective Teaching 


30 


33.0 


• ^ 


.6 


Teacher Clarity Training 


4 


3.3 


0 


0.0 


Handicapping toreness 


0 


0.0 


8 


5.0 


Cultural Awareness 


0 


0.0 


1 


.6 


Rope Course 


0 


0.0 


1 


.6 


In-class Session/ Interaction 


4 


4.4 


0 


0.0 


Other 


5 


5.5 


0 


0.0 


Non-codeable* 




2.2 


5 


2.5 




91 


100.0 


161 


100.1** 



♦Note: The total codeable events (89 for Education 450 and 159 for Education 451) 
are used In the following tables. 

' Rounding error 

. M 1} 




Table 5 



Type of Event 
;ibil1ty 



(Teacher Responsibility or Area of Decision Making) 



Educ 450 Educ 451 

K % K % 

Planning 27 30.3 35 22.0 

Teaching 41 46.1 38 23.9 

Classroom Control; Teacher- Student 3 3.4 • 31 19.5 
Relationship'" 

Student Characteristics 7 7.9 40 25.2 

Professionalism 2 2.2 12 7.5 

Other Ji 10.1 3 1.9 

89 100.0 159 100.0 



Table 6 



Category of Event 
(Specific Situations, Behaviors, or Outcomes Within Types of Events) 



K 



PLANNING 

1. Use of curriculum guides i. 

2. fbtch of content and strategies 1 

3. Time for careful preparation 19 

4. Changing plafig 0 

5. Space utilization 0 

6. Use of a written plan 2 

7. Use of an outline plan 1 

8. Use of own creativity 3 



27 

ERIC 





Educ 450 




Educ 


451 


% of 


% of 




% of 


%'of 


Total 


Planning 




Total 


Planning 


Events 


Events 


K 


Events 


Events 


1.1 


3.7 


2 


* 

1.3 


5.7 


11 


3.7 


6 


3.8 


17.1 


21.3 


70.4 


9 


5.7 


2^.7 " 


0.0 


0.0 


7 


4.4 


20.0 


0.0 


0.0 


0 


0.0 


0.0 


2.2 


7.4 ^ 


3 


1.9 


8.6 


1.1 


3.7 


2 


1.3 


5.7 


3.4 


11.1 


6 


3.8 


17.1 


30.3 


100.0 


3F 


22.2 


99.9 
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Table 6 (continued) 



Educ 450 



Educ 451 



TEAQiING 

9/Learner attention 

10, Strategies Involving learners 

11, Successful lesson 

12, Unsuccessful lesson 

13, Unresponsive learners 

14, Unclear directions 

15, Inrpact of competition 

16, Impact of tests 

17, Impact of feedback 

18, Lack of knowledge of content 

19, Contrwerslal content 

20; Transfer cf Neaching skills 





% of 
TSial 


% of 
Teaching 




% of 

lolil 


% of 
teachi f 


i 


Events 


Events 


JC 


tvents 


Events 


0 


0.0 


0.0 


2 


1.3 


5.3 


2 


2.2 


4.9 


7 


4.4 


18.4 


14 


15.7 


34.1 


16 


10.1 


42.1 


3 


3.4 


7.3 


1 


.6 


2.6 


0 


0,0 


0.0 


2 


1.3 


5.3 


3 


3.4 


7.3 


2 


1.3 


5.3 


0 


0.0 


0,0 


1 


.6 


2.6 


1 


1.1 


2.4 


2 


1.3 


5.3 


14 


15.7 


34.1 


1 


.6 


2.6 


1 


1,1 


2.4 


2 


1.3 


5.3 


2 


2.2 


4.9 


2 


1,3 


5.3 


1 


1.1 


2.4 


0 


0.0 


0.0 






99.8 • 


IS 


23,9 


100,1 



K 

CLASSROW CONTROL; 
TEACHER-STUDENT RELATIONSHIPS 

21, Setting rules and expectations 1 

22, Being fair and consistent 0 

23, Uncoopertatlve students 0 

24, Misbehaving students 0 
Z5t Disrespect of students ^ 0 

26. Immoral Intentions or acts 0 

27. Negative and abusive discipline 1 

28. Positive relnforcenent, 1_ 

recognition 

3 



« of X of % of % of 

Total Control Total Control . 

Events Events K Events Events 



1.1 


33.3 


7 


4.4 


22.6 


0.0 


0.0 


3 


1.9 


9.7 


0.0 


0.0 


0 


0.0 


0.0 


0.0 


0.0 


5 


3.2 


16.1 


0.0 


0.0 


0 


0.0 


0.0 


0.0 


0.0 


1 


.6 


3.2 


1.1 


33.3 


8 


5.0 


25.8 


1.1 


33.3 


-I 


4.4 


22.6 


3,3 


99.9 


31 


19.4 


100.0 



IC 

STUDENT CHARACTERISTICS 

29. Unexpected learner character^ 6 

Utics 

30. Not knowing individuals 0 

31. Providing for specfal needs 1 

32. Meeting \ tried needs 0 

33. Learner background 0 

34. Khowledgeable learners 0 



% of % of % of % of 

Total Char . Total Char . 

Events Events K Events Events 



6.7 


85.7 


25 


15.7 


62.5 


0.0 


0.0 


1 


.6 


2.5 


1.1 


14.3 


6 


3.8 


15.0 


0.0 


0.0 


3 


1.9 


7.5 


0.0 


0.0 


4 


2.5 


10.0 


0.0 


0.0 


1 


.6 


2.5 




100.6 




25.1 


100.0 



Table 6 (continued) 



Educ 450 



Educ 451 



PROFESSIONAL I»1 

35. Labeling of learners 

36. Problems with cooperating 

teacher feedback 

37. Veteran teacher discouragement 

38. Expert teacher modeling 

39. Lack of expert teacher modeling 

40. Disagreement with teacher goals, 

beliefs, actions 



K 

Q 
1 

0 
1 
0 
0 



^OTHER EVENTS 

41. Effect of group cooperation 

42. Effect of negative attitude 

43. Providing space, facilities 
Other 



3 
1 
0 
5 
? 

5? 



% of 
Tofal 



% of 
Profess, 



% of 

Totil 



% of 
Profess. 



Events 


Events 


K 


Events 


Events 


0.0 


0.0 


0 


0.0 


0.0 


1.1 


50. C 


0 


0.0 


0.0 


0.0 


0.0 


2 


1.3 


16.7 


1.1 


50.0 


2 


1.3 


16.7 


n n 


fl n 

W. V 


C 

9 


1 1 
J. 1 




0.0 


0.0 


J. 


1.9 


25.0 


2.e 


100.00 


12 


7.6 


100.1 


% of 


% of 




35 Of 


r of 


Total 


other 




Total 


Other 


Events 


Events 


K 


Events 


Events 


3.4 


33.3 


1 


.6 


33.3 


1.1 


11.1 


0 


0.0 


0.0 


0.0 


0.0 


2 


1.3 


66.6 


5.6 


55.6 


0 


0.0 


0.0 


10.1 


lOO.O 


J 


TT? 


90 


99.8 




m 


100.1 
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Table 7 



Htqh FreQuency Categories 
(Event reported five or more times ) 



Type 0. Event 
Planning : ' 



450 

tategory of Event 
*1, Time for careful preparation 



451 

Category of Event 

*1. Time for careful preparation 

2. Changing plans 

3. Using creativity 

4. Matching content to 
strategies 



Teaching 


*2. Successful lesson 
3. Impact of Evaluative feedback 


*5. Successful lesson ^ 
6. Strategies Involving 
learners 


-( 

Classroom Control 
and Teacher-Pupil 
Relationships: 




7. Negative discipline 
H, Positive reinforcement 
9, Setting rules 
10. Reacting to misbehavior 


» 

Student 

Characteristics: 


*4. Unexpected learner ^ 
characteristics 


*11. Unexpected learner 

chairacterlsltcs 
12. Providing for special needs 


Professionalism: 




13. Lack of expifrt teacher 
modeling 


Other 


5. Other 





♦Frequently occurring categories In both Education 450 and 451. 



Table 8 
Affect of Event 





Educ 


450 


Educ 451 




K 


% 


i i 


Positive 


69 


77.5 


89 56.0 


Neutral 


7 


7.9 


14 8.8 


Negative 


13 


14.6 


56 35.2 




100.0 


m 100.0 
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Appendix A 

1/83 

The Critical Event In PI 

The use of the Critical Event record in PI has spme similarities to the 
Experience Report Form (ERF) used throughout the Freshman Early Experiencing 
Program. In PI we are Interested In having you formulate conclusions about 
the professional events which had a significant impact on you. 

PI Critical Events are the parts of professional experiences which have 
particular importance and meaning to you. Such events will frequently evoke 
feelings and thoughts which can be formulated into personal theories to guide 
actions In educational settings. j 

In reporting a Critical Event it is important to describe a specific event 
and to separate description from Interpretations and conclusions. 

Specifying an event . Focus on situations that occur within your experiences 
In the field, lab, classroom, or individual work. Decide on the particular 
situations and the factors influencing them which are most pertinent to your 
feelings and thoughts. 

m 

Separating description from interpretations and conclusions .* Accounts of what 
happened in situations often contain a mixture of information and facts (low 
inference; description) and value statements, observer Inferences and observer 
characterizations (high inference; judgments)'. The report form is divided 
into two sections. In the description section, statements should contain the 
ofoser/ed circumstances and behaviors. In the judgment sectiori, statements 
should contain your feelings, thoughts, and conclusions. 



Example 

Description of Event 



Judgments of Event 



For my second RTL, I prepared a 
written plan and referred to it 
about 8 times during the 10 minute 
lesson. 

I spent approximately 3 hours pre- 
paring the lesson; twice as long 
as for niy first RTL. 

I rejected 3 approaches before I 
came up with a way to teach which 
hadn't been tried before in 450. 
For my first RTL I used the first 
idea I had come up with. 



I felt more relaxed and a great deal 
more confident than I did during my 
first RTL. I was more organized and 
felt that the lesson pi on helped 
considerably. 

This lesson was creative because I 
thought of a different way to do It. 
The preparation required more time; 
but it was worth it because the lesson 
was creative and successful. 

Overall Conclusi on: I shoul<| explore 
beyond my first iTeas of way^ to teach 
bikause I want to be creative and suc- 
cessful in my teaching. 



Your instructor will ask you to complete a number of Critical Event forms during PI. 
At the end of the quarter select the most significant event and give your instructor 
a copy. This one Critical Event form will be filed with the Student Infofmation System. 

^Duncan, James K. Climate for Learning: Evaluation Component . Bloomington, 
111.: Phi Delta Kappa, 1980, '■ 
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?A, CRITICAL EVENT REPORT FORM 

Kame 

SS N 

Date 

Course Ho, 



Describe an event which had a significant impact on you. First, describe the 
factual circumstances and behaviors of the event. Second, state your feelings, 
thoughts and conclusions resulting from the event. 



Description of the Event 


Judgments of thet 


/ent 









/ 



f 



Overall Conclusion; 
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(g. mteh ef wntm iwd itrateqiw ta context 



03. Spwdlaq tlw for eireftil prwratlw 



04. Sttuatfont rwifHno ctwwa cf plans 



OS. Sbm» uUKxatlen 



OS. UM of wri ttM Plan! to orwrtM. to f Prwared 



07. utt of >r1»f notwt «it11wt p'.tw *<»11» taatftlng 



08. tfw of cnMt<»1ty, ptrtwwl ItfMP 



Set 3; Catgwrlw of Ewttt (e«»tfBM«» 

18. iMCttm to l^t of pofUtirt rtfnforewrti 
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